Context-aware Mission Control for Astronaut-Robot Collaboration

Space robot assistants are envisaged as semi-autonomous co-workers deployed to lighten the workload of astronauts in cumbersome and dangerous situations. In view of this, this work considers the prospects on the technology requirements for future space robot operations, by presenting a novel mission control concept for close astronaut-robot collaboration. A decentralized approach is proposed, in which an astronaut is put in charge of commanding the robot, and a mission control center on Earth maintains a list of authorized robot actions by applying symbolic, geometric, and context-specific filters. The concept is applied to actual space robot operations within the METERON SUPVIS Justin experiment. In particular, it is shown how the concept is utilized to guide an astronaut aboard the ISS in its mission to survey and maintain a solar panel farm in a simulated Mars environment

Building on the Moon Using Additive Manufacturing: Discussion of Robotic Approaches

We present two concepts for building structures on the moon. The first involves an all-purpose robot with exchangeable tools, able to excavate regolith, produce slurry and print a structure. The second involves a team of robots: one excavator, one sintering using sunlight, and another gathering the sintered bricks and building a structure. ESA's Space Resources Strategy sets the challenge of achieving human presence at the Moon, sustained by local resources, by 2040, which includes in-situ manufacturing and construction. We take a realistic look at the feasibility of these concepts and their component technologies, and their challenges with respect to the state of the art and expected technological readiness in the next 2-5 years

Model Mediated Teleoperation with a Hand-Arm Exoskeleton in Long Time Delays Using Reinforcement Learning

Telerobotic systems must adapt to new environmental conditions and deal with high uncertainty caused by long-time delays. As one of the best alternatives to human-level intelligence, Reinforcement Learning (RL) may offer a solution to cope with these issues. This paper proposes to integrate RL with the Model Mediated Teleoperation (MMT) concept. The teleoperator interacts with a simulated virtual environment, which provides instant feedback. Whereas feedback from the real environment is delayed, feedback from the model is instantaneous, leading to high transparency. The MMT is realized in combination with an intelligent system with two layers. The first layer utilizes Dynamic Movement Primitives (DMP) which accounts for certain changes in the avatar environment. And, the second layer addresses the problems caused by uncertainty in the model using RL methods. Augmented reality was also provided to fuse the avatar device and virtual environment models for the teleoperator. Implemented on DLR's Exodex Adam hand-arm haptic exoskeleton, the results show RL methods are able to find different solutions when changes are applied to the object position after the demonstration. The results also show DMPs to be effective at adapting to new conditions where there is no uncertainty involved

Model Mediated Teleoperation with a Hand-Arm Exoskeleton in Long Time Delays Using Reinforcement Learning

elerobotic systems must adapt to new environmental conditions and deal with high uncertainty caused by long-time delays. As one of the best alternatives to human-level intelligence, Reinforcement Learning (RL) may offer a solution to cope with these issues. This paper proposes to integrate RL with the Model Mediated Teleoperation (MMT) concept. The teleoperator interacts with a simulated virtual environment, which provides instant feedback. Whereas feedback from the real environment is delayed, feedback from the model is instantaneous, leading to high transparency. The MMT is realized in combination with an intelligent system with two layers. The first layer utilizes Dynamic Movement Primitives (DMP) which accounts for certain changes in the avatar environment. And, the second layer addresses the problems caused by uncertainty in the model using RL methods. Augmented reality was also provided to fuse the avatar device and virtual environment models for the teleoperator. Implemented on DLR's Exodex Adam hand-arm haptic exoskeleton, the results show RL methods are able to find different solutions when changes are applied to the object position after the demonstration. The results also show DMPs to be effective at adapting to new conditions where there is no uncertainty involved

Exodex Adam—A Reconfigurable Dexterous Haptic User Interface for the Whole Hand

Applications for dexterous robot teleoperation and immersive virtual reality are growing. Haptic user input devices need to allow the user to intuitively command and seamlessly “feel” the environment they work in, whether virtual or a remote site through an avatar. We introduce the DLR Exodex Adam, a reconfigurable, dexterous, whole-hand haptic input device. The device comprises multiple modular, three degrees of freedom (3-DOF) robotic fingers, whose placement on the device can be adjusted to optimize manipulability for different user hand sizes. Additionally, the device is mounted on a 7-DOF robot arm to increase the user’s workspace. Exodex Adam uses a front-facing interface, with robotic fingers coupled to two of the user’s fingertips, the thumb, and two points on the palm. Including the palm, as opposed to only the fingertips as is common in existing devices, enables accurate tracking of the whole hand without additional sensors such as a data glove or motion capture. By providing “whole-hand” interaction with omnidirectional force-feedback at the attachment points, we enable the user to experience the environment with the complete hand instead of only the fingertips, thus realizing deeper immersion. Interaction using Exodex Adam can range from palpation of objects and surfaces to manipulation using both power and precision grasps, all while receiving haptic feedback. This article details the concept and design of the Exodex Adam, as well as use cases where it is deployed with different command modalities. These include mixed-media interaction in a virtual environment, gesture-based telemanipulation, and robotic hand–arm teleoperation using adaptive model-mediated teleoperation. Finally, we share the insights gained during our development process and use case deployments

A Newcomer's Guide to the Challenges of a Complex Space-to-Ground Experiment, With Lessons from Analog-1

An astronaut controlling a complex robot on the surface of earth from the ISS. This is exactly what we have done in ANALOG-1. Luca Parmitano teleoperated a rover in a moon-analogue geological mission scenario. On first sight the primary technical challenges seem to be the design of the robotic systems for space and ground. On a second look - with the perspective of using the system with an astronaut on the ISS in loop with an operations team in different ground centers - the scope and challenges drastically increase. In this paper we take a look behind the scenes, and gives insights which could guide future payload developers going on a similar endeavour. This paper outlines the Analog-1 experiment, itself, what it aimed to achieve, and how it was done, and uses it as a case study to outline the challenges and solutions a project team and particularly the payload developer - will have to overcome when designing an ISS experiment. This article may be especially insightful and a good starting point for those from a small research team at a university or other research institution with budget and time pressure. We will present it from the payload developers perspective and on concrete examples of the payloads we flown

Model-Augmented Haptic Telemanipulation: Concept, Retrospective Overview, and Current Use Cases

Certain telerobotic applications, including telerobotics in space, pose particularly demanding challenges to both technology and humans. Traditional bilateral telemanipulation approaches often cannot be used in such applications due to technical and physical limitations such as long and varying delays, packet loss, and limited bandwidth, as well as high reliability, precision, and task duration requirements. In order to close this gap, we research model-augmented haptic telemanipulation (MATM) that uses two kinds of models: a remote model that enables shared autonomous functionality of the teleoperated robot, and a local model that aims to generate assistive augmented haptic feedback for the human operator. Several technological methods that form the backbone of the MATM approach have already been successfully demonstrated in accomplished telerobotic space missions. On this basis, we have applied our approach in more recent research to applications in the fields of orbital robotics, telesurgery, caregiving, and telenavigation. In the course of this work, we have advanced specific aspects of the approach that were of particular importance for each respective application, especially shared autonomy, and haptic augmentation. This overview paper discusses the MATM approach in detail, presents the latest research results of the various technologies encompassed within this approach, provides a retrospective of DLR's telerobotic space missions, demonstrates the broad application potential of MATM based on the aforementioned use cases, and outlines lessons learned and open challenges

Cartesian Impedance Control on Five-Finger Dexterous Robot Hand DLR-HIT II with Flexible Joint

This paper presents an impedance controller for five-finger dexterous robot hand DLR-HIT II, which is derived in Cartesian space. By considering flexibility in finger joints and strong mechanical couplings in differential gear-box, modeling and control of the robot hand are described in this paper. The model-based friction estimation and velocity observer are carried out with an extended Kalman filter, which is implemented with parameters estimated by Least Squares Method. The designed estimator demonstrates good prediction performance, as shown in the experimental results. Stability analysis of the proposed impedance controller is carried out and described in this paper. Impedance control experiments are conducted with the five-finger dexterous robot hand DLR-HIT II in Cartesian coordinates system to help study the effectiveness of the proposed controller with friction compensation and hardware architecture

Ubiquitous user interface design for space robotic operation

Autonomous robotic systems are approaching the maturity level to be able to support astronauts in space and on planetary surfaces. Commanding these robots with existing input devices and command modalities cannot cope with their increased capabilities and limits the usability for the astronaut. This paper presents alternative user interface (UI) concepts for the use with ubiquitous devices such as smartwatches and tablet computers. In particular, it is proposed to command an autonomous space robot assistant on a low to medium level of autonomy using a smartwatch. When commanding the robot on a high level of autonomy, a tablet computer can be utilized with object-centered task-level commands. The respective interaction concepts designed to make best use of the input devices are presented in detail. A comparison of the proposed interfaces and the presentation of the use of the tablet computer user interface in the METERON SUPVIS Justin ISS experiment concludes the paper